Skip to content

[tests] Route Maven and Gradle resolution through the dnceng mirror - #12199

Merged
jonathanpeppers merged 25 commits into
dotnet:mainfrom
jonathanpeppers:jonathanpeppers-fix-cfsclean-maven-feeds
Jul 29, 2026
Merged

[tests] Route Maven and Gradle resolution through the dnceng mirror#12199
jonathanpeppers merged 25 commits into
dotnet:mainfrom
jonathanpeppers:jonathanpeppers-fix-cfsclean-maven-feeds

Conversation

@jonathanpeppers

@jonathanpeppers jonathanpeppers commented Jul 21, 2026

Copy link
Copy Markdown
Member

Pull Request
title and
description
should follow the
commit-messages.md workflow documentation, and in particular should include:

  • Useful description of why the change is necessary.
  • Links to issues fixed - N/A
  • Unit tests

Summary

CFSClean reports test jobs that resolve Maven dependencies and Gradle plugins directly from public services. CI agents are network-isolated and can only reach the anonymous dotnet-public-maven Azure Artifacts feed, which mirrors both Maven Central and Google Maven.

This routes every test Maven and Gradle resolution through that feed. Crucially, it does so unconditionally rather than only when RUNNINGONCI is set — a CI-only code path means a local run exercises different URLs than CI, so a test can pass locally and then fail in the pipeline. Making local and CI identical is the same principle applied to the mirror-seeding tool below.

Test infrastructure

  • Xamarin.ProjectTools exposes a single TestEnvironment.DotNetPublicMaven constant. Test download URLs and <AndroidMavenLibrary Repository="..."> metadata are built from it directly, replacing the old GetTestDownloadUrl/GetMavenRepository helpers and their table of public-repository prefixes. A missing prefix in that table used to fail silently on CI only.
  • AndroidGradleProject writes a settings.gradle.kts that applies the shared eng/gradle repository scripts, and copies the repository Gradle wrapper instead of downloading an alternate distribution.
  • Java.Interop java-source-utils no longer declares mavenCentral() independently, and parallel Kotlin Gradle builds are isolated so they don't race.

Mirror seeding

eng/gradle/mirror-dependencies.ps1 now runs Gradle exactly the way CI does — anonymously, with the configuration cache enabled — instead of loading an Azure Artifacts credential-provider plugin behind an env var. That plugin forced the seeding run to differ from the real run, so lazily-resolved Kotlin and lint classpaths resolved differently and the feed was seeded with the wrong packages. Authentication moved out of Gradle entirely: 401 URLs are re-fetched over plain HTTP with an Azure DevOps OAuth token, which makes the feed's upstream connector cache them for anonymous reads. eng/gradle/credential-provider.gradle is removed. A -MavenArtifact mode seeds coordinates directly for tests that don't use Gradle.

Follow-on fixes

  • TestEnvironment.IsWindows/IsMacOS/IsLinux are annotated [SupportedOSPlatformGuard] so CA1416 can see through them. The properties were always correct at runtime; the analyzer just couldn't recognize an arbitrary bool as a platform guard. Annotating the shared helper fixes every current and future caller instead of rewriting call sites.
  • Xamarin.Google.Android.InstallReferrer is bumped to 2.2.0.8 to match Facebook SDK 18.3.0, which requires com.android.installreferrer:2.2.

Testing

  • Built Xamarin.ProjectTools, Xamarin.Android.Build.Tests, and MSBuildDeviceIntegration — 0 errors.
  • All 17 MavenDownloadTests pass locally, including the three that download through the mirror. Because the mirror is now used locally too, this exercises the same URLs CI does.
  • Verified every artifact referenced by a test resolves on the feed (HEAD → 200), so nothing 401s on CI.
  • Routing all downloads through a repository URL would leave the Repository="Central"/"Google" shorthands unexercised, so that mapping is extracted to MavenDownload.GetKnownRepository and covered by new tests that need no network and therefore run under CI isolation.
  • Passed 106 Java.Interop Maven tests; java-source-utils resolves through the feed.
  • GradleFBProj passes on device after the InstallReferrer bump.
  • Full pipeline green apart from the pre-existing, unrelated JnienvArrayMarshaling.GetObjectArray JNI peer-registration flake.

Route test Maven and Gradle resolution through dotnet-public-maven when
RUNNINGONCI is set while retaining public repositories for local development.
Teach generated Gradle projects to use the shared repository configuration and
extend the mirror helper for tests that resolve Maven files without Gradle.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Copilot AI review requested due to automatic review settings July 21, 2026 19:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates test infrastructure and Gradle test projects to ensure Maven/Gradle dependency resolution on CI uses the anonymous dotnet-public-maven Azure Artifacts mirror (per RUNNINGONCI=true) rather than reaching out to public repositories, while keeping existing public-repo behavior for local development.

Changes:

  • Centralizes CI detection and Maven URL rewriting in Xamarin.ProjectTools.TestEnvironment, and routes DownloadedCache downloads through it.
  • Updates multiple test suites to select the Maven mirror on CI and to skip/adjust assertions for behaviors that differ on the mirror (e.g., uncached artifacts returning 401).
  • Aligns generated and in-repo Gradle settings.gradle(.kts) with shared eng/gradle/*-repositories.gradle scripts; extends mirror-dependencies.ps1 to support direct Maven coordinate seeding.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs Ensures AndroidMavenLibrary items use the CI mirror via TestEnvironment when running on CI.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Common/TestEnvironment.cs Adds RUNNINGONCI detection and helpers to select the mirror and rewrite Maven Central/Google download URLs on CI.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Common/DownloadedCache.cs Rewrites download URLs through TestEnvironment.GetTestDownloadUrl() to enforce mirror usage on CI.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Android/AndroidGradleProject.cs Generates settings.gradle.kts using shared eng/gradle repository scripts and avoids downloading alternate Gradle distributions on CI.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/MavenDownloadTests.cs Switches to TestEnvironment mirror selection on CI; skips tests whose expected 404 behavior differs on the mirror; removes null-forgiving usage in touched code.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BindingBuildTest.cs Sets Maven repository metadata via TestEnvironment to enforce mirror usage on CI.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidGradleProjectTests.cs Skips multi-Gradle-version test cases on CI to avoid public Gradle distribution downloads.
external/Java.Interop/tools/java-source-utils/settings.gradle Uses shared eng/gradle repository scripts for plugin/dependency resolution.
external/Java.Interop/tools/java-source-utils/build.gradle Removes per-project repositories { mavenCentral() } to rely on shared settings-based repository policy.
external/Java.Interop/tests/Java.Interop.Tools.Maven-Tests/Extensions/MavenProjectResolver.cs Selects the mirror repository on CI for Java.Interop Maven tests while keeping existing local behavior.
eng/gradle/mirror-dependencies.ps1 Adds -MavenArtifact mode to seed the mirror directly for tests that fetch Maven artifacts without Gradle.
.github/instructions/gradle.instructions.md Documents the new direct Maven-coordinate seeding mode for the mirror helper script.

Put each importing test project Kotlin intermediates under its own obj directory so concurrent solution builds cannot remove another compiler classpath snapshot.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Select the Gradle executable already installed on CI agents so Java.Interop test builds do not download a distribution from services.gradle.org. Local builds continue using the repository wrapper.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Hosted Windows and macOS agents do not provide Gradle on PATH. Remove the preinstalled-Gradle override so Java.Interop jobs use their checked-in wrapper again while the isolated Gradle distribution awaits feed seeding.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Run restore out of process before BootstrapTasks and workloads builds. This ensures generated NuGet path properties are imported on clean agents instead of evaluating MonoUnixNative.targets with an empty PkgMono_Unix path.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
@jonathanpeppers

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@jonathanpeppers

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Authenticate Azure Artifacts Maven requests with the Azure DevOps OAuth token as a Basic credential, and disable Gradle configuration caching while the credential provider runs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
@jonathanpeppers

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

jonathanpeppers and others added 10 commits July 22, 2026 10:22
Run mirror resolution anonymously with configuration caching enabled so it discovers the same lazy Kotlin and lint classpaths as CI. Seed each 401 through the authenticated HTTP path instead of loading a Gradle credential provider.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Pin BindFacebook to the mirrored 18.3.0 release and teach the mirror helper to seed payloads that Gradle probes with HEAD as well as GET.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Use the same mirrored Facebook Android SDK version in GradleFBProj as the host BindFacebook test so the device lane resolves the validated anonymous dependency closure.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Copy the checked-in Gradle wrapper instead of relying on gradle init, and keep all Facebook fixture dependencies aligned on the mirrored version.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Keep the missing-wrapper regression check in the existing generated app test instead of maintaining a separate test for test infrastructure.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Build the BootstrapTasks and workloads projects entirely out of process and capture timestamped binlogs alongside the other Prepare logs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Briefly explain why each generated gradle.properties option is enabled.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
Document why generated Gradle projects use the checked-in wrapper.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d21fe48a-552e-44ca-8c0b-739e7c2b7129
CA1416 does not understand the `TestEnvironment.IsWindows` guard, so
the `File.GetUnixFileMode`/`File.SetUnixFileMode` calls broke the
build. Use `OperatingSystem.IsWindows ()` which the analyzer
recognizes as a platform guard.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
Revert the `OperatingSystem.IsWindows ()` workaround and instead
annotate `TestEnvironment.IsWindows`/`IsMacOS`/`IsLinux` with
`[SupportedOSPlatformGuard]`. The properties always worked at runtime;
only the CA1416 analyzer could not see through them, and it only
recognizes `OperatingSystem.Is*()`, `RuntimeInformation.IsOSPlatform`
or members carrying a platform-guard attribute. Annotating them fixes
every current and future caller instead of just this one.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
…fsclean-maven-feeds

# Conflicts:
#	external/Java.Interop/tests/Xamarin.Android.Tools.Bytecode-Tests/kotlin-gradle.targets
jonathanpeppers added a commit that referenced this pull request Jul 27, 2026
…12231)

The `Xamarin.Android Nightly` build fails in `make jenkins` -> `make prepare`:

    /Users/builder/.dotnet/sdk/10.0.302/Microsoft.Common.CurrentVersion.targets(5397,5):
      error MSB3030: Could not copy the file "/runtimes/linux-x64/native/libMono.Unix.so"
      because it was not found.
      [.../build-tools/Xamarin.Android.Tools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks.csproj]

The leading bare `/` shows `$(PkgMono_Unix)` evaluated to an empty string. The
package content is present; the path property was simply never set.

`$(PkgMono_Unix)` comes from `GeneratePathProperty="true"` on the `Mono.Unix`
`PackageReference` in `MSBuildReferences.projitems`. NuGet writes it into
`obj\*.nuget.g.props`, which only a *fresh project evaluation* picks up.
`build-tools/scripts/Prepare.proj` invoked BootstrapTasks with:

    <MSBuild Projects="...BootstrapTasks.csproj" Targets="Restore;Build" />

The `MSBuild` task evaluates a project once and runs both targets against that
single project instance, so unlike the `-restore` switch it never re-evaluates
between `Restore` and `Build`. With no pre-existing `obj/`, the static `None`
item in `MonoUnixNative.targets` therefore resolved to a root-relative path.

Before 94e4833 the `prepare` target ran
`dotnet build Xamarin.Android.BootstrapTasks.sln`, whose implicit restore does
re-evaluate, which is why this only regressed now and only on clean agents --
a warm `obj/` on a dev machine hides it.

Build these projects out of process with `dotnet build` instead, matching the
approach in #12199 (the change to `Prepare.proj` here is identical, so the two
will not conflict). `dotnet build` restores in a separate evaluation and also
writes a `.binlog` per project, so the next `make prepare` failure is
diagnosable from CI artifacts. `PrepareOpenJDK` in `DotNet.targets` gets the
same treatment, since it had the identical latent bug.

Also add `_ValidateMonoUnixPackagePath` to `MonoUnixNative.targets` so an unset
`$(PkgMono_Unix)` reports what is actually wrong rather than an `MSB3030` on a
nonsense path. It runs `BeforeTargets="PrepareForBuild"` so it fires before
`_CopyOutOfDateSourceItemsToOutputDirectory` can attempt the copy.

Verified locally with a clean `bin/` and `obj/`:

- the old `Targets="Restore;Build"` pattern reproduces the failure
  (`D:\runtimes\...` on Windows, the same empty-prefix bug as CI's `/runtimes/...`)
- with the guard in place that same pattern now reports only:
  ``error : `$(PkgMono_Unix)` is not set. Build this project with `dotnet build` so NuGet-generated props are evaluated after restore.``
- `dotnet build` of BootstrapTasks succeeds with 0 warnings and 0 errors, and
  copies the real `libMono.Unix.so` (120768 bytes, matching the file in the
  `Mono.Unix` package) plus `libMono.Unix.dylib`

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
jonathanpeppers and others added 2 commits July 27, 2026 08:21
Apply the instructions to `*.gradle.kts` too, since Kotlin DSL settings
files use the same shared repository config. Show the Kotlin variant and
note that the relative depth to the repo root varies by project.

Document how tests stay on the mirror: generated Gradle projects apply
the shared config and reuse the repository wrapper, while non-Gradle
downloads route through `TestEnvironment.GetMavenRepository` and
`GetTestDownloadUrl`. Add a don't for Gradle credential providers, now
that authentication lives only in `mirror-dependencies.ps1`.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
jonathanpeppers and others added 2 commits July 27, 2026 16:20
GradleFBProj moved to the mirrored Facebook SDK 18.3.0, which requires
`com.android.installreferrer:installreferrer:2.2`. The project still
referenced `Xamarin.Google.Android.InstallReferrer` 1.1.2.6, which
binds installreferrer 1.1.2, so `_VerifyJavaDependencies` failed the
device build with XA4242.

Reference 2.2.0.8, whose `artifact_versioned` metadata declares
installreferrer 2.2 and satisfies the new dependency closure.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
Test Maven downloads were rewritten to the dnceng `dotnet-public-maven` feed
only when `RUNNINGONCI` was set. That made local runs take a different code
path than CI: a test could pass locally against `repo1.maven.org` and then
fail on CI, where the network is isolated and only the mirror is reachable.
A missing entry in the prefix table failed the same way, silently.

The feed mirrors both Maven Central and Google Maven and is readable
anonymously, so there is no reason to gate on CI. Use it unconditionally:

- Replace `TestEnvironment.GetTestDownloadUrl` and the `publicMavenRepositories`
  prefix table with a public `TestEnvironment.DotNetPublicMaven` constant, and
  build test URLs from it directly.
- Replace `TestEnvironment.GetMavenRepository ("Central"|"Google")` with the
  same constant.

Routing every test through a repository URL leaves the `Repository="Central"`
and `"Google"` shorthands unexercised, so extract the mapping into
`MavenDownload.GetKnownRepository` and cover it directly. That test needs no
network, so it also runs under CI's isolation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
@jonathanpeppers jonathanpeppers changed the title [tests] Isolate Maven resolution on CI [tests] Route Maven and Gradle resolution through the dnceng mirror Jul 28, 2026
@jonathanpeppers

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Android PR Reviewer completed successfully!

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Review — routing Maven/Gradle resolution through the dnceng mirror

Reviewed the 25 changed files (test infra, MSBuild task refactor, Gradle settings, and the mirror-seeding script). The change is well-scoped and the test-side reasoning is sound: hoisting the DotNetPublicMaven constant into TestEnvironment, replacing hard-coded repo1.maven.org / maven.google.com URLs, and extracting MavenDownload.GetKnownRepository so the Central/Google shorthands stay covered without network is a clean approach. The [SupportedOSPlatformGuard] annotations on TestEnvironment.IsWindows/IsMacOS/IsLinux are the right fix for CA1416, and dropping the null-forgiving ! operators in favor of explicit throws matches repo conventions.

Findings:

  • 💡 2 suggestions (inline): a RUNNINGONCI gating inconsistency in MavenProjectResolver that runs counter to this PR's "local == CI" principle, and a robustness nit on CopyGradleWrapper's File.Copy error surface.

Nothing blocking from a correctness standpoint — no bugs, leaks, or security issues found. Removing the Gradle credential-provider plugin in favor of anonymous resolution + HTTP seeding in mirror-dependencies.ps1 is a nice simplification, and switching the seeding auth to Basic (:$token) is the correct Azure DevOps pattern.

CI: GitHub statuses on the head commit (1e71df2) are still pending/not yet reported, and the Azure DevOps dotnet-android pipeline result isn't visible here. The PR description notes only a pre-existing unrelated JnienvArrayMarshaling.GetObjectArray flake. Please confirm the pipeline is green before merging — this PR's whole premise is CI-network-isolation behavior, so a green run on the isolated agents is the real validation.

Verdict: ⚠️ Needs Changes — only the two 💡 items to consider; primary gate is confirming CI is green.

Generated by Android PR Reviewer for #12199 · 130.6 AIC · ⌖ 18.8 AIC · ⊞ 6.9K
Comment /review to run again

jonathanpeppers and others added 4 commits July 28, 2026 10:59
MavenProjectResolver still selected the dnceng feed only when RUNNINGONCI was
set, which contradicts the rest of these tests: they now go through the mirror
unconditionally so a local run exercises the same URLs as CI. Drop the gate.

Verified with a cold Maven cache so the downloads are real: all 106 tests pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
The file's only remaining difference from main was a deleted blank line at
EOF, left over from an earlier commit in this branch. The functional change
that commit made here has since been reverted, so drop the whitespace noise
and take the file out of the diff entirely.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 79b5be27-8ca9-42a6-ac4f-10d69c200075
@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Jul 28, 2026
@jonathanpeppers
jonathanpeppers merged commit 466da4a into dotnet:main Jul 29, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants